The present invention relates to a scene-change-point detecting method, apparatus and computer program. More particularly the present invention relates to a scene-change-point detecting method, apparatus and computer program for detecting a scene change point in a moving-picture encoding system so as to find the beginning (head) of a compressed/coded video signal or video data for each scene and a moving-picture editing/displaying method, apparatus and computer program for editing and displaying a moving picture by using the scene-change-point detecting method, apparatus and computer program.
Since a digital video signal has a large amount of data, a secondary storage device is required for handling video data extending over a long period of time. For this reason, in recent years, a Moving Picture Experts Group (MPEG) format is proposed and used as a system for reducing the amount of video data by compressing a digital moving picture. Details of this format are disclosed in International Standards for Multimedia Coding, by H. Yasuda, Maruzen publishing, 1991 or ISO/IEC International Standard Information Technology-Generic Coding of Moving Pictures and Associated Audio for Digital Storage Media at up to about 1.5 Mbit/s, 1993.
According to the MPEG, a picture is compressed by using three techniques: intra-picture compression for producing an xe2x80x9cIxe2x80x9d picture (Intra-Picture), a forward-predicted compression for producing a xe2x80x9cPxe2x80x9d picture (Predictive-Picture) and a forward/backward-predicted compression for producing a xe2x80x9cBxe2x80x9d picture (Bidirectionally Predictive-Picture). An xe2x80x9cIxe2x80x9d picture is divided into blocks each comprising 16xc3x9716 picture elements. A Discrete Cosine Transformation (DCT) is carried out for each block. The DCT concentrates picture information on coefficients of low-frequency components. The values are further quantized by considering the fact that the sense of sight of a human being is dim to the high-frequency components. Information compressed by using these two processings is then coded by using the Huffman table.
A xe2x80x9cPxe2x80x9d picture is produced by carrying out differential-picture compression with reference to a timewise preceding xe2x80x9cIxe2x80x9d picture or xe2x80x9cPxe2x80x9d picture and a timewise succeeding xe2x80x9cIxe2x80x9d picture or xe2x80x9cPxe2x80x9d picture. First of all, a picture to be compressed is divided into blocks each comprising 16xc3x9716 picture elements. Then, either intra-picture compression, differential-picture compression or no data compression (or skipped compression) is selected for each of the blocks. The intra-picture compression is compression in a block carried out in the same way as that for producing an xe2x80x9cIxe2x80x9d picture. In the differential-picture compression, motion compensation is carried out on the image of a block to be compressed with respect to the picture elements of the reference-picture, allowing a motion compensating vector to be determined. If the motion compensating vector is the same as that of a block preceding the block to be compressed, the data of the block to be compressed can be skipped.
A xe2x80x9cBxe2x80x9d picture is produced by differential-picture compression with reference to a timewise preceding xe2x80x9cIxe2x80x9d picture or xe2x80x9cPxe2x80x9d picture and a timewise succeeding xe2x80x9cIxe2x80x9d picture or xe2x80x9cPxe2x80x9d picture. Much like a xe2x80x9cPxe2x80x9d picture, the picture to be compressed is divided into blocks each comprising 16xc3x9716 picture elements. Then, either intra-picture compression, differential-picture compression or no data compression (or skipped compression) is selected for each of the blocks. The selection is carried out in the same way as a xe2x80x9cPxe2x80x9d picture. The MPEG allows the compression performance to be enhanced by using such differential-picture compression among pictures.
There is a conventional technology for extracting change points of a digital moving picture. The conventional technology is used for moving-picture editing and finding the head of a scene. Since a compressed moving picture is a picture having differential information as described above, the conventional technology can not be applied to such a compressed moving picture as it is.
There has been proposed a method for extracting change points of data to be compressed which includes typically the steps of: creating a reduced picture of each compressed picture by using only an average value of small block units of the compressed picture, finding correlation values of luminance and chrominance histograms in the same-way as the ordinary picture processing for the reduced picture, and detecting a scene change point from variations in correlation value along the time axis.
Details of the above described method are disclosed in Japanese Patent Laid-open No. Hei 6-231254. This method offers a merit of a processing speed higher than that of a method whereby compressed moving-picture data is completely decoded before extracting change points.
In another method, a scene change is detected from the coding quantity of a compressed picture during compression of the picture. Details of this method are disclosed in Japanese Patent Laid-open No. Hei 6-133305. According to this method, a scene change point can be detected with ease while a picture is being compressed.
In the former method whereby correlation values of luminance and chrominance histograms are found by using only an average value of small block units, since picture processing is carried out in the extraction of a change point after simplified decoding, scene detection takes a long time, making it difficult to detect a change point quickly.
The latter method whereby the quantity of coding is compared with that of the preceding picture is, on the other hand, effective for compression of using only an xe2x80x9cIxe2x80x9d picture such as the case with the motion Joint Picture Experts Group (JPEG). However, the method can not be applied to the MPEG wherein the adopted coding technique varies from block to block.
An object of the present invention to provide a scene-change-point detecting method, apparatus and computer program and a moving-picture editing/displaying method, apparatus and computer program capable of automatically extracting a scene change from a moving picture and editing, displaying as well as searching a moving picture with ease.
In order to achieve the object described above, the present invention provides a scene-change-point detecting method, apparatus and computer program for detecting a scene change point in a moving picture coded according to a moving-picture coding process. The moving-picture coding process codes the moving picture by dividing the moving picture into groups of pictures each having a first picture to undergo intra-picture compression, a plurality of second pictures each to undergo forward differential-picture compression wherein differences from a timewise preceding encoded picture are compressed and a plurality of third pictures each to undergo forward/backward differential-picture compression wherein differences from a timewise preceding encoded picture and a timewise succeeding encoded picture are compressed; dividing each of the pictures into blocks each having a predetermined size; and encoding each of the blocks by one of the intra-picture compression, the forward differential-picture compression and the forward/backward differential-picture compression.
The scene-change-point detecting method, apparatus and computer program includes the steps of or performs the functions of: counting the number of blocks in a picture completing the differential-picture compression, the number of blocks in the picture completing the intra-picture compression and the number of blocks in the picture completing no compression; and detecting a scene change point for a number of blocks constituting the picture by using the block counts. It should be noted that the group of picture is also referred to hereafter simply as a GOP.
In addition, in the scene-change-point detecting method, apparatus and computer program, information required in detection of a scene change point derived from the number of blocks in a picture completing the differential-picture compression, the number of blocks in the picture completing the intra-picture compression and the number of blocks in the picture completing no compression, is recorded as user data.
Further, in order to achieve the object described above, the present invention provides a moving picture editing/displaying method, apparatus and computer program for editing and displaying a moving picture by decoding, editing and displaying a stored, compressed and coded moving picture. The moving-picture editing/displaying method, apparatus and computer program includes the steps of or performs the functions of: storing a picture position specified in the moving picture, if any, as user data; searching the user data for the specified picture position in a process-of decoding the stored, compressed and coded moving picture; detecting a scene change point preceding the picture position and a scene change point succeeding the picture position by using a scene-change-point detecting method, apparatus and computer program; and displaying pictures starting with a picture at the scene change point preceding the picture position and ending with a picture at the scene change point succeeding the picture position.